1. The Field of the Invention
This invention relates generally to generating images for a virtual set, and more particularly to a method and apparatus by which an entire viewport definition for a computer generated image is computed from a camera's pan, tilt, zoom, and focus sensor information.
2. The Background Art
Calculating virtual sets is known in the art of computer generated imaging. A virtual set combines scenes viewed and recorded by a camera with scene elements (such as backdrops) from some other source. The scenes recorded by the camera are typically either moving or stationary objects and the scenes are photographed in front of a plain, colored wall; typically, the wall being colored blue or green. Special equipment or processes are then used to combine the camera scenes with the other scene elements. The combination is usually accomplished using compositing techniques.
For example, a typical application of compositing techniques involves combining a camera scene with a background scene, wherein the camera scene is a live person in the foreground and the background scene is a photograph of some other scene (a "synthetic scene"). Using these techniques, everything in the camera scene which is the color of the studio backdrop (e.g., blue) is replaced with the synthetic scene. Preferrably, the combined image will then appear as though the person is located in the background scene.
Rather than just placing the camera scene in front of the background scene (but clearly not blending into the background scene), often it is desirable to make the camera scene appear as if it was actually part of the background scene. In such cases, it is necessary to know, with considerable precision, enough information about the camera to generate the scene that the camera is "seeing" if the background scene were really part of the camera's environment. These calculations require that the image generator know what a camera is "seeing" so that the image generator can create the corresponding scene from a virtual database. The required camera information includes knowing the XYZ position, the orientation (pan, tilt, and roll), field of view (zoom), and focus.
Precise calculations of the camera information has been a problem for years. Some systems involve setting up cameras using a tape measure to measure distances for each position of the camera. For example, if the camera is on a pedestal, measuring the location of the pedestal can be useful information when determining camera position. Then, rather than calculating the angular position of the camera, the camera scene is manually matched, by observation, with the synthetic scene. These calculations introduce other problems because they may not take into account nodal point offset from the pan/tilt axis origin which results in registration errors in a composite image. Additionally, the calculations may simply not allow any nodal point offset from the pan/tilt axis intersection. The result is that the center of gravity is put off of the tilt axis, causing motors to burn power while dynamically holding the camera in position. In addition, even if nodal position may be accurately computed, the calculation may be accurate for only one field of view.
As appreciated by those skilled in the art, the available calibration techniques are very tedious, often requiring measurements using an optical workbench and data points recorded manually. In fact, some techniques for properly moving a camera on a virtual set require up to 45 minutes for each camera move. When cameras are moved multiple times during the day on the virtual set, considerable time is lost and the importance of more rapidly calibrating the necessary virtual set data is appreciated. Of current interest is a method and apparatus for more rapidly computing an entire viewport definition for the calculation and generation of virtual images.